Modular and Hierarchical Specializationin Neural Networks
نویسندگان
چکیده
Modularity and hierarchy are fundamental notions in structured system design. By subdividing a large and unstructured problem into smaller and tractable chunks, design automation becomes possible. In this paper we discuss the use of modularity and hierarchy for functional specialization during the development of neural networks. We study the behavioral differences and requirements for back-propagation training of feed-forward networks. Further we illustrate that a deliberate mix of hierarchically imposed evaluation functions will improve network accuracy and learning speed.
منابع مشابه
معرفی شبکه های عصبی پیمانه ای عمیق با ساختار فضایی-زمانی دوگانه جهت بهبود بازشناسی گفتار پیوسته فارسی
In this article, growable deep modular neural networks for continuous speech recognition are introduced. These networks can be grown to implement the spatio-temporal information of the frame sequences at their input layer as well as their labels at the output layer at the same time. The trained neural network with such double spatio-temporal association structure can learn the phonetic sequence...
متن کاملHierarchical modular structure enhances the robustness of self-organized criticality in neural networks
One of the most prominent architecture properties of neural networks in the brain is the hierarchical modular structure. How does the structure property constrain or improve brain function? It is thought that operating near criticality can be beneficial for brain function. Here, we find that networks with modular structure can extend the parameter region of coupling strength over which critical...
متن کاملSustained Activity in Hierarchical Modular Neural Networks: Self-Organized Criticality and Oscillations
Cerebral cortical brain networks possess a number of conspicuous features of structure and dynamics. First, these networks have an intricate, non-random organization. In particular, they are structured in a hierarchical modular fashion, from large-scale regions of the whole brain, via cortical areas and area subcompartments organized as structural and functional maps to cortical columns, and fi...
متن کاملOptimal Hierarchical Modular Topologies for Producing Limited Sustained Activation of Neural Networks
An essential requirement for the representation of functional patterns in complex neural networks, such as the mammalian cerebral cortex, is the existence of stable regimes of network activation, typically arising from a limited parameter range. In this range of limited sustained activity (LSA), the activity of neural populations in the network persists between the extremes of either quickly dy...
متن کاملPattern Recognition for Industrial Security using the Fuzzy Sugeno Integral and Modular Neural Networks
We describe in this paper the evolution of modular neural networks using hierarchical genetic algorithms for pattern recognition. Modular Neural Networks (MNN) have shown significant learning improvement over single Neural Networks (NN). For this reason, the use of MNN for pattern recognition is well justified. However, network topology design of MNN is at least an order of magnitude more diffi...
متن کامل